Exposition and Analysis of a Suffix Sorting Algorithm

This paper focuses on the suffix sorting algorithm of Maniscalco [10], which at the time of writing is available only as C++ source code on the Internet. We will refer to the program as MSufSort. MSufSort computes the Inverse Suffix Array (ISA) of an input string, which is equivalent to computing the Suffix Array (converting one to the other is discussed in section 8). Recall that for i ∈ [0..n − 1], ISA[i] gives the lexicographic rank of the suffix x[i..n − 1] amongst all the other suffixes of the string x[0..n − 1]. Experiments summarized in [10] suggest that MSufSort outperforms the fastest known suffix sorting programs, while using little extra space aside from the 4n bytes to hold the suffix array and the n bytes for the input string (in the terms of [11] it would be lightweight). It is also purported to perform well on periodic strings, which are known to be catastrophic worst cases for some algorithms. This paper addresses the need for a more formal examination of what appears to be a very robust suffix sorter. We examine and describe the inner workings of the algorithm, and try to explain why MSufSort performs well by analyzing its asymptotic behavior. As published in [10], the MSufSort source code crosses several classes and files and is not easy absorb in a single sitting. The code presented in this paper constitutes a complete rewrite of the original as just a few C functions, and is included not as an optimization, but rather to aid explanation of the approach. After introducing some notation in Section 2, the basic algorithm is described in Section 3 before two powerful heuristics are introduced in Sections 4 and 5. Sections 6 and 7 consider time and space usage. Section 8 discusses ISA to SA transformation. In Section 9 we extensively test MSufSort and compare its performance to that of other leading suffix sorters. Possible areas for future work are outlined in Sections 10 and 11, and brief conclusions are offered in Section 12. It is assumed the reader is familiar with the concept of suffix sorting and its applications, particularly the Burrows-Wheeler Transformation (BWT).